Image processing system, projector, program, information storage medium and image processing method

ABSTRACT

To provide an image processing system and the like which can correct a distortion of a projected image through only a single sensor, a projector has a calibration image information generating section which generates image information used to display a rectangular calibration image; an image projection section which projects the calibration image onto a projection target, based on the image information; a sensing section which senses a region including the projected calibration image plane and generates sensing information; a projection area information generating section which generates shortest-distance projection area information based on the sensing information in such a state that a distance between the projection section and the projection target is different from the distance in an actual operating condition, and generates actual projection area information based on the sensing information in the actual operating condition; and a correction information generating section which generates image distortion correction information, based on the shortest-distance projection area information and actual projection area information.

Japanese Patent Application No. 2003-273358, filed on Jul. 11, 2003, ishereby incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

The present invention relates to an image processing system, projector,program, information storage medium and image processing method whichcan correct the distortion of an image.

When an image is projected by an image projection device such as aprojector and if the optical axis of the projected light is notcoincident with a projection target such as a screen, the projectedimage may be distorted to create a so-called keystone distortion fromimage distortions in the vertical and horizontal directions.

For such a reason, the image projection device must project a properimage after the distortion has been eliminated.

However, the general image projection device having an image distortioncorrecting function includes an inclination sensor which can correct theimage distortion in the vertical direction, but not the image distortionin the horizontal direction.

In order to correct the distortion in the horizontal direction, a usermust indicate four corner points in a screen by use of a mouse or thelike so that the image projection device can semi-automatically correctthe image distortion in the horizontal direction, based on the user'sindications. It is troublesome for the user to indicate the four cornerpoints in the screen through the mouse or the like.

To solve such a problem, for example, Japanese Patent Laid-OpenApplication No. 2001-61121 has proposed a projector device whichcomprises a projection plane capturing means for capturing the shape ofthe projection plane by associating coordinates of the projector's imagewith coordinates of the camera's image based on an image taken by acamera, the taken image being projected when a pattern image isdisplayed, and determining three-dimensional coordinate of the positionwhere the pattern image is projected through triangulation, and an imagecorrection means for performing corrections for inclination andmagnification in the original inputted image according to the sensedshape of the projection plane.

Japanese Patent Laid-Open Application No. 2001-61121 also describes thata known three-dimensional shape measuring device can be used asprojection plane capturing means and that the projection plane sensingmeans are not essential for the invention thereof.

However, the known three-dimensional shape measuring device requires twocameras, for example, when the triangulation is to be carried out. Thismakes the structure complicated and increases the manufacturing cost.

Additionally, such a technique of detecting the shape of the projectiontarget such as a screen to process the image as in the Japanese PatentLaid-Open Application No. 2001-61121 cannot be applied to a wall. Thus,projection targets to which such a technique can be applied will belimited.

BRIEF SUMMARY OF THE INVENTION

The present invention is made in view of the above-mentioned problem andmay provide an image processing system, projector, program, informationstorage medium and image processing method, all of which can correct anydistortion of an projected image by use of a single sensor for measuringthe distance and angle between a reference projection point and aprojection target.

An image processing system or projector according to an aspect of thepresent invention includes:

-   -   a calibration image information generating means for generating        image information used to display a rectangular calibration        image;    -   a projection means for projecting the calibration image onto a        projection target, based on the image information;    -   a sensing means for sensing a region including the projected        calibration image through a sensing plane and generating sensing        information;    -   a projection area information generating means for generating        temporary projection area information which indicates        coordinates of four corners of the calibration image on the        sensing plane, based on the sensing information in such a state        that an entire portion of the calibration image is included        within a sensing range of the sensing means and under such a        state that a distance between the projection means and the        projection target is different from the distance in an actual        operating condition, and for generating actual projection area        information which indicates the coordinates of the four corners        of the calibration image in the sensing plane, based on the        sensing information in the actual operating condition;    -   a correction information generating means for generating image        distortion correction information; and    -   a distortion correction means for correcting a distortion of the        image, based on the image distortion correction information,        wherein the correction information generating means has        following functions of:    -   deriving an inter-coordinate distance between each of the        coordinates of the four corners included in the temporary        projection area information and corresponding one of the        coordinates of the four corners included in the actual        projection area information;    -   deriving a projection distance on an optical axis of the        projection means between the projection means and each of the        four corners of the calibration image on the projection target,        based on projection distance data in which the inter-coordinate        distance is associated with the projection distance and based on        the inter-coordinate distance;    -   deriving three-dimensional coordinates of at least three of the        four corners of the calibration image in a three-dimensional        space for the projection means to process an image, based on the        projection distance and the half angle-of-view in the projection        means;    -   deriving an angle formed by the optical axis of the projection        means and the projection target, based on the three-dimensional        coordinates; and    -   generating the image distortion correction information, based on        the derived angle.

An image processing system or projector according to another aspect ofthe present invention includes:

-   -   a calibration image information generating section which        generates image information used to display a rectangular        calibration image;    -   a projection section which projects the calibration image onto a        projection target, based on the image information;    -   a sensing section which senses a region including the projected        calibration image through a sensing plane and generates sensing        information;    -   a projection area information generating section which generates        temporary projection area information which indicates        coordinates of four corners of the calibration image on the        sensing plane, based on the sensing information in such a state        that an entire portion of the calibration image is included        within a sensing range of the sensing section and under such a        state that a distance between the projection section and the        projection target is different from the distance in an actual        operating condition, and generates actual projection area        information which indicates the coordinates of the four corners        of the calibration image in the sensing plane, based on the        sensing information in the actual operating condition;    -   a correction information generating section which generates        image distortion correction information; and    -   a distortion correction section which corrects a distortion of        the image, based on the image distortion correction information,        wherein the correction information generating section has        following functions of:    -   deriving an inter-coordinate distance between each of the        coordinates of the four corners included in the temporary        projection area information and corresponding one of the        coordinates of the four corners included in the actual        projection area information;    -   deriving a projection distance on an optical axis of the        projection section between the projection section and each of        the four corners of the calibration image on the projection        target, based on projection distance data in which the        inter-coordinate distance is associated with the projection        distance and based on the inter-coordinate distance;    -   deriving three-dimensional coordinates of at least three of the        four corners of the calibration image in a three-dimensional        space for the projection section to process an image, based on        the projection distance and the half angle-of-view in the        projection section;    -   deriving an angle formed by the optical axis of the projection        section and the projection target, based on the        three-dimensional coordinates; and    -   generating the image distortion correction information, based on        the derived angle.

A computer-readable program according to a further aspect of the presentinvention causes a computer to function as:

-   -   a calibration image information generating means for generating        image information used to display a rectangular calibration        image;    -   a projection means for projecting the calibration image onto a        projection target, based on the image information;    -   a sensing means for sensing a region including the projected        calibration image through a sensing plane and generating sensing        information;    -   a projection area information generating means for generating        temporary projection area information which indicates        coordinates of four corners of the calibration image on the        sensing plane, based on the sensing information in such a state        that an entire portion of the calibration image is included        within a sensing range of the sensing means and under such a        state that a distance between the projection means and the        projection target is different from the distance in an actual        operating condition, and for generating actual projection area        information which indicates the coordinates of the four corners        of the calibration image in the sensing plane, based on the        sensing information in the actual operating condition;    -   a correction information generating means for generating image        distortion correction information; and    -   a distortion correction means for correcting a distortion of the        image, based on the image distortion correction information,        wherein the correction information generating means has        following functions of:    -   deriving an inter-coordinate distance between each of the        coordinates of the four corners included in the temporary        projection area information and corresponding one of the        coordinates of the four corners included in the actual        projection area information;    -   deriving a projection distance on an optical axis of the        projection means between the projection means and each of the        four corners of the calibration image on the projection target,        based on projection distance data in which the inter-coordinate        distance is associated with the projection distance and based on        the inter-coordinate distance;    -   deriving three-dimensional coordinates of at least three of the        four corners of the calibration image in a three-dimensional        space for the projection means to process an image, based on the        projection distance and the half angle-of-view in the projection        means;    -   deriving an angle formed by the optical axis of the projection        means and the projection target, based on the three-dimensional        coordinates; and    -   generating the image distortion correction information, based on        the derived angle.

An information storage medium according to a still further aspect of thepresent invention stores a computer-readable program which causes acomputer to function as:

-   -   a calibration image information generating means for generating        image information used to display a rectangular calibration        image;    -   a projection means for projecting the calibration image onto a        projection target, based on the image information;    -   a sensing means for sensing a region including the projected        calibration image through a sensing plane and generating sensing        information;    -   a projection area information generating means for generating        temporary projection area information which indicates        coordinates of four corners of the calibration image on the        sensing plane, based on the sensing information in such a state        that an entire portion of the calibration image is included        within a sensing range of the sensing means and under such a        state that a distance between the projection means and the        projection target is different from the distance in an actual        operating condition, and for generating actual projection area        information which indicates the coordinates of the four corners        of the calibration image in the sensing plane, based on the        sensing information in the actual operating condition;    -   a correction information generating means for generating image        distortion correction information; and    -   a distortion correction means for correcting a distortion of the        image, based on the image distortion correction information,        wherein the correction information generating means has        following functions of:    -   deriving an inter-coordinate distance between each of the        coordinates of the four corners included in the temporary        projection area information and corresponding one of the        coordinates of the four corners included in the actual        projection area information;    -   deriving a projection distance on an optical axis of the        projection means between the projection means and each of the        four corners of the calibration image on the projection target,        based on projection distance data in which the inter-coordinate        distance is associated with the projection distance and based on        the inter-coordinate distance;    -   deriving three-dimensional coordinates of at least three of the        four corners of the calibration image in a three-dimensional        space for the projection means to process an image, based on the        projection distance and the half angle-of-view in the projection        means;    -   deriving an angle formed by the optical axis of the projection        means and the projection target, based on the three-dimensional        coordinates; and    -   generating the image distortion correction information, based on        the derived angle.

An image processing method according to a yet further aspect of thepresent invention includes:

-   -   causing an image projection device to project a rectangular        calibration image onto a projection target, sensing a region        including the projected calibration image through a sensing        plane, and generating first projection area information which        indicates coordinates of four corners of the calibration image        in the sensing plane, when a positional relationship between the        image projection device and the projection target becomes a        first state in which an entire portion of the calibration image        is included within a sensing range;    -   causing the image projection device to project the calibration        image onto the projection target, sensing the region including        the projected calibration image through the sensing plane, and        generating second projection area information which indicates        the coordinates of the four corners of the calibration image in        the sensing plane, when the positional relationship between the        image projection device and the projection target becomes a        second state in which an entire portion of the rectangular        calibration image is included within the sensing range, the        second state being different from the first state;    -   deriving an inter-coordinate distance between each of the        coordinates of the four corners included in the first projection        area information and the corresponding one of the coordinates of        the four corners included in the second projection area        information;    -   deriving a projection distance on an optical axis of a projected        light between the image projection device and each of the        coordinates of the four corners of the calibration image on the        projection target, based on the projection distance data in        which the inter-coordinate distance is associated with the        projection distance and based on the inter-coordinate distance;    -   deriving three-dimensional coordinates of at least three of the        four corners of the calibration image in a three-dimensional        space for the image projecting device to process an image, based        on the projection distance and the half angle-of-view in the        projected light;    -   deriving an angle formed by the optical axis of the projected        light and the projection target, based on the three-dimensional        coordinates;    -   generating image distortion correction information, based on the        derived angle; and    -   correcting image distortion, based on the generated image        distortion correction information.

In accordance with the present invention, the image processing systemand the like can correct the distortions in the projected image by useof only a single sensor for measuring the distance and angle between thereference projection point and the projection target.

In accordance with the present invention, moreover, the image processingsystem and the like can be applied to various kinds of projectiontargets such as walls, resulting in improvement of its versatility,since the distortions in the projected image can be corrected withoutusing a frame of a screen and the like.

With these image processing system, projector, program and informationstorage medium, the correction information generating means may havefollowing functions of:

-   -   deriving a normal vector in the projection target, based on the        three-dimensional coordinates; and    -   deriving an angle formed by the optical axis of the projection        means and the projection target, based on the normal vector and        a directional vector of the optical axis of the projection        means.

When the angle formed by the optical axis of the projected light and theprojection target is derived, this image processing method may include:

-   -   deriving a normal vector in the projection target, based on the        three-dimensional coordinates; and    -   deriving an angle formed by the optical axis of the projected        light and the projection target, based on the normal vector and        a directional vector of the optical axis of the projected light.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

FIG. 1 is schematic view showing a state when an image is projected.

FIG. 2 is a functional block diagram of a projector according to oneexample of an embodiment of the present invention.

FIG. 3 is a hardware block diagram illustrating a projector according tothis example of this embodiment.

FIG. 4 is a flow chart illustrating a flow of image processing accordingto this example of this embodiment.

FIG.5 is a schematic view of a sensing plane according to this exampleof this embodiment.

FIG. 6 is a schematic view illustrating, in a plan view, a state ofprojection according to this example of this embodiment.

FIG. 7 is a schematic view illustrating, in a side view, a state ofprojection according to this example of this embodiment.

FIG.8 is a schematic view illustrating a projection distance accordingto this example of this embodiment.

FIG. 9 is a schematic view illustrating the data structure of theprojection distance according to this example of this embodiment.

FIG. 10 is a schematic view illustrating four corner coordinates in acorrected projection area according to this example of this embodiment.

FIG. 11 is correcting data illustrating four corner coordinates in acorrected projection area according to this example of this embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENT

In the description below, the present invention is applied to aprojector which performs a distortion correction of an image, by way ofexample, with reference to the accompanying drawings. Note that theembodiments described hereunder do not in any way limit the scope of theinvention defined by the claims laid out herein. Note also that all ofthe elements of these embodiments should not be taken as essentialrequirements to the means of the present invention.

Overall System

FIG. 1 is schematic view showing a state when an image is projected.

A projector 20 which is a kind of an image processing system projects animage toward a screen 10 which is a kind of a projection target. Thus, aprojected image 12 is displayed on the screen 10.

In this embodiment, the projector 20 is not arranged directly in frontof the screen 10. For such a reason, a distortion (e.g., a so-calledkeystone distortion) is occurred in the projected image 12.

In this embodiment, a sensor 60 which is part of a sensing means sensesa region including the projected image 12. The projector 20 then derivesa projection distance between the reference projection position of theprojector 20 and each of four corners of the projected image 12 based onthe sensing information from the sensor 60, and derives an angle betweenthe optical axis in the projected light from the projector 20 and thescreen 10.

The projector 20 further determines a distortion of the projected image12 based on the projection distances and corrects input image signals tocorrect the distortions in the projected image 12.

Functional Blocks

Next, the functional blocks of the projector 20 for implementing such afeature will be described.

FIG. 2 is a functional block diagram of the projector 20 according toone example of this embodiment.

The projector 20 comprises a signal input section 110 for inputtingimage signals, a distortion correcting section 130 for correcting theinputted image signals so that the image distortion is corrected, asignal output section 160 for outputting the corrected image signals, animage projecting section 190 for projecting an image based on the imagesignals, the image projecting section 190 being a kind of projectionmeans, and a calibration image information generating section 170 forgenerating calibration image information.

The projector 20 also comprises a sensing section 180 for sensing aregion including the projected image 12 through the imaging sensingplane to generate sensing information, a projection area informationgenerating section 150 for determining the region of the projected image12 in the sensing plane of the sensor 60, based on the sensinginformation and a correction information generating section 120 forgenerating distortion correction information. The sensor 60 is includedin the sensing section 180.

The image projecting section 190 comprises a spatial light modulator192, a drive section 194 for driving the spatial light modulator 192, alight source 196 and a lens 198.

The drive section 194 drives the spatial light modulator 192 based onthe image signals from the signal output section 160. The imageprojecting section 190 projects light from a light source 196 throughthe spatial light modulator 192 and lens 198.

The following means can be applied so that the functions of therespective sections in the projector 20 can be implemented into acomputer.

FIG. 3 is a hardware block diagram illustrating a projector according tothis example of this embodiment.

The signal input section 110 may be implemented, for example, by an A/Dconverter 930 or the like; the distortion correcting section 130 may beimplemented, for example, by an image processing circuit 970, RAM 950,CPU 910 or the like; the signal output section 160 may be implemented,for example, by a D/A converter 940 or the like; the correctioninformation generating section 120, projection area informationgenerating section 150 and calibration image information generatingsection 170 may be, for example, by the image processing circuit 970,RAM 950 or the like; the sensing section 180 may be implemented, forexample, by a CCD sensor, a CMOS sensor, an RGB sensor or the like; thespatial light modulator 192 may be implemented, for example, by a liquidcrystal panel 920, a ROM 960 for storing a liquid crystal light valvedriver for driving the liquid crystal panel 920 or the like.

These sections can be configured to mutually deliver the informationtherebetween through a system bus 980.

Moreover, the part or whole of each of these sections may be implementedin a hardware manner such as circuits or in a software manner such asdrivers.

The computer may implement the function of the projection areainformation generating section 150 according to a program that has beenstored in and is read out of an information storage medium 900, theprogram being operative for causing the computer to function as theprojection area information generating section 150 and the like.

Such an information storage medium 900 may be accomplished, for example,by CD-ROM, DVD-ROM, ROM, RAM, HDD or the like. The program may be readout of the information storage medium 900 through either of the contactor non-contact type reading mode.

In place of the information storage medium 900, the aforementionedfunctions can be implemented by downloading a program or the like forimplementing them from a host device or the like through a transmissionchannel.

Flow of Image Processing

Next, the flow of image processing using these sections will bedescribed.

FIG. 4 is a flow chart illustrating a flow of image processing accordingto this example of this embodiment.

A manufacturer for the projector 20 has determined, prior to delivery ofthe projector 20, the relationship between the distances on the opticalaxis of the projected light between the reference projection position ofthe image projecting section 190 and four corners of the projected image12 and the distances between coordinates in the sensing plane of thesensor 60 and stored it in the correction information generating section120 as projection distance data.

Prior to delivery of the projector 20, the manufacturer has also locatedthe screen 10 at a shortest-distance position (first or temporaryposition) at which the whole calibration image (or the projected image12) falls within the sensing range in the sensor 60 under a conditionthat the projector 20 is positioned directly in front of the screen 10.The projector 20 projects a calibration image toward the screen 10,derives the coordinates of the four corners of the projected image 12 onthe sensing plane, and stores them as shortest-distance projection areainformation (step S1).

More particularly, the calibration image information generating section170 generates image information for an all-white monochromaticcalibration image (i.e., white-colored image as a whole). The signaloutput section 160 outputs the digital signals of the image informationtoward the image projecting section 190.

The image projecting section 190 projects an all-white calibration imageonto the screen 10, based on the digital signals. Thus, the all-whitecalibration image is displayed on the screen 10.

The sensing section 180 senses a region including the projected image 12through the imaging sensing plane to generate sensing information. Here,the sensing information is one that indicates an image signal valuewhich can generate a luminance value, XYZ value or the like for eachpixel in the sensor 60. The XYZ value used herein is a kind of imagesignal value which is a device-independent color based on theInternational Standard defined by the Commission Internationale del'Eclairage (CIE).

Next, the sensing plane and projection state will be described.

FIG. 5 is a schematic view illustrating the sensing plane 17 accordingto this example of this embodiment. FIG. 6 is a schematic viewillustrating, in a plan view, a projection state according to thisexample of this embodiment. FIG. 7 is a schematic view illustrating, ina side view, a state of projection according to this example of thisembodiment.

The sensing plane 17 shown in FIG. 5 is a plane at which the sensor 60senses the image and also a region in which the sensing information ofthe sensor 60 is schematically shown in rectangular form. Specifically,the sensing plane 17 corresponds to, for example, that of a CCD sensor.

If the screen 10 is located at the minimum distance in which the entirecalibration image falls within the sensing range in the sensor 60, theprojected image 14 corresponds to the shortest-distance projection area15 in the sensing plane 17, which is surrounded by four vertexes A0 toD0.

The projected image 12 in the actual operating environment correspondsto the actual projection area 13 in the sensing plane 17, which issurrounded by four vertexes A to D.

The projection area information generating section 150 generatesshortest-distance projection area information (first or temporaryprojection area information) representing the coordinates of the fourvertexes A0 to D0 of the shortest-distance projection area 15 in thesensing plane 17, based on the sensing information for the projectedimage 14 from the sensing section 180.

The manufacturer has also sensed images through the sensor 60 whilemoving the projector 20 away from the screen 10 within a range ofprojection between the minimum and maximum distances. The maximumdistance used herein is the longest distance on the optical axis of theprojected light, which quality is assured by the maker, when theprojector 20 is positioned directly in front of the screen 10.

The correction information generating section 120 determines theprojection area information within the range of projection between theminimum and maximum distances and the projection distance at each pointof time. The correction information generating section 120, thencomputes inter-coordinate distances at the respective one of the sensedprojection distances, each of the inter-coordinate distancesrepresenting a deviation (e.g., dots) between the coordinate of each ofthe four corners in the sensing plane 17 of the projected image 12 andthe coordinate of each of the four corners of the projection image 12when it is projected at the minimum distance.

The correction information generating section 120 further generates andstores a two-dimensional lookup table in which the inter-coordinatedistances are associated with the projection distance as projectiondistance data.

By the above-mentioned procedure, the manufacturer can deliver theprojector 20 which has stored the projection distance data in which theinter-coordinate distance between the projections at the minimum andmaximum distances is associated with the projection distance as well asthe shortest-distance projection area information.

When a user first uses the projector 20 in the actual operatingenvironment, the projection area information generating section 150generates the actual projection area information (second projection areainformation) representing the coordinates of the four vertexes A to D ofthe actual projection area 13 in the sensing plane 17, based on thesensing information for the projected image 12 from the sensing section180 in the actual operating environment (step S2).

The correction information generating section 120 then derives aninter-coordinate distance in the sensing plane 17 between each of thevertexes of the actual projection area 13 and corresponding one of thevertexes of the shortest-distance projection area 15, based on theshortest-distance projection area information and the actual projectionarea information (step S3).

The correction information generating section 120 then derives theprojection distance based on the inter-coordinate distance andprojection distance data (step S4).

FIG.8 is a schematic view illustrating a projection distance accordingto this example of this embodiment. FIG. 9 is a schematic viewillustrating the data structure of the projection distance according tothis example of this embodiment.

It is now assumed that a position at which a projection lens section 50including the lens 198 emits the projected light (reference projectionposition) is an origin and that the lens plane is X-Y plane while theoptical axis of the projected light is Z-axis. In this case, thecorrection information generating section 120 derives a projectiondistance Ad, Bd, Cd or Dd on the Z-axis between a shortest-distancepoint between each of the vertexes A, B, C or D in the projected image12 and the Z-axis, and the origin.

More specifically, the correction information generating section 120derives a projection distance (Ad, Bd, Cd or Dd) to each of the fourcorners of the projected image 12, based on the derived inter-coordinatedistances and the projection distance data indicative of therelationship between the derived inter-coordinate distances and theprojection distances. For example, in such projection distance data asshown in FIG. 9, the projection distance is equal to 1.5 m if theinter-coordinate distance is zero dots. The projection distance data maybe common to all the four vertexes or individual from one vertex toanother.

It is further assumed that each half angle-of-view in the projectionlens section 50 is θR on the right side; θL on the left side; θU on theupper side; or θD on the lower side as shown in FIGS. 6 and 7.

In this case, the projector coordinates A′, B′, C′ and D′ of the fourcorners of the projected image 12 (coordinates in the three-dimensionalspace for processing the spatial light modulator 192) are:

-   -   A′(A′x, A′y, A′z)=(Ad*tan(θL), Ad*tan(θD), Ad);    -   B′(B′x, B′y, B′z)=(Bd*tan(θL), Bd*tan(θU), Bd);    -   C′(C′x, C′y, C′z)=(Cd*tan(θR), Cd*tan(θU), Cd); and    -   D′(D′x, D′y, D′z)=(Dd*tan(θR), Dd*tan(θD), Dd).

The correction information generating section 120 computes the normalvector N (Nx, Ny, Nz) in the screen 10 on which the projected image 12is displayed by use of three points among these projector coordinates.For example, when three points A′, C′ and D′ are used,

-   -   Nx=(D′y−C′y)*(A′z−D′z)−(D′z−C′z)*(A′y−D′y);    -   Ny=(D′z−C′z)*(A′x−D′x)−(D′x−C′x)*(A′z−D′z); and    -   Nz=(D′x−C′x)*(A′y−D′y)−(D′y−C′y)*(A′x−D′x).

The correction information generating section 120 also derives an angle(θx in the horizontal direction or θy in the vertical direction) whichis formed by the normal vector N and a directional vector (0, 0, 1) ofthe optical axis of the projection lens section 50 (step S5). This anglecorresponds to an angle formed by the screen 10 and the optical axis ofthe projected light from the projector 20.

The correction information generating section 120 further derivescoordinates of four corners E, F, G and H in the corrected projectionarea by searching the coordinates of the four corners in the correctedprojection area from correcting data based on this angle (i.e., θx orθy) (step S6).

FIG. 10 is a schematic view illustrating four corner coordinates in acorrected projection area according to this example of this embodiment.FIG. 11 is correcting data illustrating four corner coordinates in acorrected projection area according to this example of this embodiment.

In the aforementioned correcting data, the values of θx and θy areassociated with the x-y coordinate of the respective one of thecoordinates of the four corners E, F, G and H in the correctedprojection area, as shown in FIG. 11. It is noted that the coordinatesof the four corners E, F, G and H in the corrected projection area arecoordinates in the spatial light modulator 192, for example.

The correction information generating section 120 then generates imagedistortion correction information by using the x-y coordinate of therespective one of the coordinates E, F, G and H in the correcting datashown in FIG. 11 (step S7), and sends them to the distortion correctingsection 130. The image distortion correction information used herein maycorrespond to information indicative of differential values between thecoordinates of the four corners of the image before and after thedistortion thereof is corrected, for example.

The distortion correcting section 130 then corrects image signals tocorrect the image distortion, based on the image distortion correctioninformation (step S8). However, the distortion correcting section 130may indirectly correct the image signals by correcting the distortioncorrecting data such as lookup table or matrix, based on the imagedistortion correction information or may directly correct the imagesignals without the use of the distortion correcting data.

Thus, the image projecting section 190 can project the image after thedistortion thereof has been corrected.

As described, according to this embodiment, the distortion of theprojected image 12 can be corrected by measuring the distance and anglebetween the point of the projection lens section 50 at which the lightis projected and the screen 10 through only a single sensor 60.

Thus, the manufacturer can provide an image processing system which isof a simpler structure produced in lower cost, in comparison with theprior art technique of detecting the three-dimensional coordinates ofthe projected image by use of a plurality of CCD cameras.

According to this embodiment, furthermore, the distortion of theprojected image 12 can be corrected without use of the four corners inthe screen 10. In other words, this embodiment may be applied to variouskinds of projection targets (e.g., white board, wall or the like).Furthermore, this embodiment may be less influenced by the color andmaterial of the projection target. According to this embodiment, thus,the versatility of the image processing system and the like can be moreimproved.

According to this embodiment, furthermore, the projector 20 can derivethe projection distance based on the inter-coordinate distance betweentwo corresponding points in the sensing plane 17 without using thedirectional information of coordinate by using the coordinates of theprojected image 12 in the sensing plane 17 when the image is projectedat the minimum distance. Thus, the projector 20 can more simply andeasily derive the projection distance to correct the distortion of theprojected image 12.

Modifications

The application of the present invention is not limited to theaforementioned embodiment.

For example, if the projector 20 has a focus adjusting function, theprojector 20 may comprise a control means for automatically adjustingthe focus based on the distance information between the projector 20 andthe screen 10, which has been generated by the correction informationgenerating section 120.

Although the aforementioned embodiment has been described as to thesensor 60 fixedly mounted in the projector 20, a positioning mechanism(e.g., an arm mechanism) for the sensor 60 may be provided so that uponcalibration, the sensor 60 senses the projected image 12 under a statethat the projection lens section 50 is located farther apart from thesensor 60.

Thus, the measurement accuracy of the sensor 60 can be more improved.

In the aforementioned embodiment, the projector 20 generates the actualprojection area information based on the sensing information in theactual operating condition and also generates the temporary projectionarea information based on the sensing information at the minimumdistance in which the calibration image falls within the sensing range.However, the temporary projection area information may be generatedbased on the sensing information which are obtained when the projector20 and screen 10 are placed under such a state that the calibrationimage falls within the sensing range and under such a state that thepositional relationship therebetween is different from the actualoperating state, rather than the minimum distance.

In the aforementioned embodiment, the two-dimensional lookup table isused as the projection distance data in which the projection distance isassociated with the inter-coordinate distance. However, a function ofinputting the inter-coordinate distance and outputting the projectiondistance may be used.

Although the aforementioned embodiment has been described as to theall-white calibration image, any monochromic calibration image otherthan white color may be used. Furthermore, the projector 20 may projectand sense an all-black calibration image (the entire image beingblack-colored) and also an all-white calibration image, compare theratios of luminance value between each pair of adjacent pixels includedin the sensing information, and determine a pixel area having itsluminance ratio more than a predetermined value as a projection area.

Thus, the projector 20 will less be influenced by any damage in thescreen 10 or by any ambient light or the like relative to the screen 10.Therefore, the projector 20 can more precisely determine the projectionarea.

Although the aforementioned embodiment uses the projector 20 as an imageprocessing system, the present invention is effective for an imageprocessing system for CRT (Cathode Ray Tube) displays, LED (LightEmitting Diode) displays, EL (Electro Luminescence) displays and otherdisplays, in addition to the projector 20.

The projector 20 may be of liquid crystal projector, a DMD (DigitalMicromirror Device) projector, for example. Here, DMD is a trademarkpossessed by Texas Instruments Inc. of the USA.

Furthermore, the function of the projector 20 may be accomplished solelyby the projector 20 or in dispersion by a plurality of decentralizedprocessing units (e.g., one projector and one PC).

1. An image processing system comprising: a calibration imageinformation generating means for generating image information used todisplay a rectangular calibration image; a projection means forprojecting the calibration image onto a projection target, based on theimage information; a sensing means for sensing a region including theprojected calibration image through a sensing plane and generatingsensing information; a projection area information generating means forgenerating temporary projection area information which indicatescoordinates of four corners of the calibration image on the sensingplane, based on the sensing information in such a state that an entireportion of the calibration image is included within a sensing range ofthe sensing means and under such a state that a distance between theprojection means and the projection target is different from thedistance in an actual operating condition, and for generating actualprojection area information which indicates the coordinates of the fourcorners of the calibration image in the sensing plane, based on thesensing information in the actual operating condition; a correctioninformation generating means for generating image distortion correctioninformation; and a distortion correction means for correcting adistortion of the image, based on the image distortion correctioninformation, wherein the correction information generating means hasfollowing functions of: deriving an inter-coordinate distance betweeneach of the coordinates of the four corners included in the temporaryprojection area information and corresponding one of the coordinates ofthe four corners included in the actual projection area information;deriving a projection distance on an optical axis of the projectionmeans between the projection means and each of the four corners of thecalibration image on the projection target, based on projection distancedata in which the inter-coordinate distance is associated with theprojection distance and based on the inter-coordinate distance; derivingthree-dimensional coordinates of at least three of the four corners ofthe calibration image in a three-dimensional space for the projectionmeans to process an image, based on the projection distance and the halfangle-of-view in the projection means; deriving an angle formed by theoptical axis of the projection means and the projection target, based onthe three-dimensional coordinates; and generating the image distortioncorrection information, based on the derived angle.
 2. The imageprocessing system as defined in claim 1, wherein the correctioninformation generating means has following functions of: deriving anormal vector in the projection target, based on the three-dimensionalcoordinates; and deriving an angle formed by the optical axis of theprojection means and the projection target, based on the normal vectorand a directional vector of the optical axis of the projection means. 3.An image processing system comprising: a calibration image informationgenerating section which generates image information used to display arectangular calibration image; a projection section which projects thecalibration image onto a projection target, based on the imageinformation; a sensing section which senses a region including theprojected calibration image through a sensing plane and generatessensing information; a projection area information generating sectionwhich generates temporary projection area information which indicatescoordinates of four corners of the calibration image on the sensingplane, based on the sensing information in such a state that an entireportion of the calibration image is included within a sensing range ofthe sensing section and under such a state that a distance between theprojection section and the projection target is different from thedistance in an actual operating condition, and generates actualprojection area information which indicates the coordinates of the fourcorners of the calibration image in the sensing plane, based on thesensing information in the actual operating condition; a correctioninformation generating section which generates image distortioncorrection information; and a distortion correction section whichcorrects a distortion of the image, based on the image distortioncorrection information, wherein the correction information generatingsection has following functions of: deriving an inter-coordinatedistance between each of the coordinates of the four corners included inthe temporary projection area information and corresponding one of thecoordinates of the four corners included in the actual projection areainformation; deriving a projection distance on an optical axis of theprojection section between the projection section and each of the fourcorners of the calibration image on the projection target, based onprojection distance data in which the inter-coordinate distance isassociated with the projection distance and based on theinter-coordinate distance; deriving three-dimensional coordinates of atleast three of the four corners of the calibration image in athree-dimensional space for the projection section to process an image,based on the projection distance and the half angle-of-view in theprojection section; deriving an angle formed by the optical axis of theprojection section and the projection target, based on thethree-dimensional coordinates; and generating the image distortioncorrection information, based on the derived angle.
 4. A projectorcomprising: a calibration image information generating means forgenerating image information used to display a rectangular calibrationimage; a projection means for projecting the calibration image onto aprojection target, based on the image information; a sensing means forsensing a region including the projected calibration image through asensing plane and generating sensing information; a projection areainformation generating means for generating temporary projection areainformation which indicates coordinates of four corners of thecalibration image on the sensing plane, based on the sensing informationin such a state that an entire portion of the calibration image isincluded within a sensing range of the sensing means and under such astate that a distance between the projection means and the projectiontarget is different from the distance in an actual operating condition,and for generating actual projection area information which indicatesthe coordinates of the four corners of the calibration image in thesensing plane, based on the sensing information in the actual operatingcondition; a correction information generating means for generatingimage distortion correction information; and a distortion correctionmeans for correcting a distortion of the image, based on the imagedistortion correction information, wherein the correction informationgenerating means has following functions of: deriving aninter-coordinate distance between each of the coordinates of the fourcorners included in the temporary projection area information andcorresponding one of the coordinates of the four corners included in theactual projection area information; deriving a projection distance on anoptical axis of the projection means between the projection means andeach of the four corners of the calibration image on the projectiontarget, based on projection distance data in which the inter-coordinatedistance is associated with the projection distance and based on theinter-coordinate distance; deriving three-dimensional coordinates of atleast three of the four corners of the calibration image in athree-dimensional space for the projection means to process an image,based on the projection distance and the half angle-of-view in theprojection means; deriving an angle formed by the optical axis of theprojection means and the projection target, based on thethree-dimensional coordinates; and generating the image distortioncorrection information, based on the derived angle.
 5. A projectorcomprising: a calibration image information generating section whichgenerates image information used to display a rectangular calibrationimage; a projection section which projects the calibration image onto aprojection target, based on the image information; a sensing sectionwhich senses a region including the projected calibration image througha sensing plane and generates sensing information; a projection areainformation generating section which generates temporary projection areainformation which indicates coordinates of four corners of thecalibration image on the sensing plane, based on the sensing informationin such a state that an entire portion of the calibration image isincluded within a sensing range of the sensing section and under such astate that a distance between the projection section and the projectiontarget is different from the distance in an actual operating condition,and generates actual projection area information which indicates thecoordinates of the four corners of the calibration image in the sensingplane, based on the sensing information in the actual operatingcondition; a correction information generating section which generatesimage distortion correction information; and a distortion correctionsection which corrects a distortion of the image, based on the imagedistortion correction information, wherein the correction informationgenerating section has following functions of: deriving aninter-coordinate distance between each of the coordinates of the fourcorners included in the temporary projection area information andcorresponding one of the coordinates of the four corners included in theactual projection area information; deriving a projection distance on anoptical axis of the projection section between the projection sectionand each of the four corners of the calibration image on the projectiontarget, based on projection distance data in which the inter-coordinatedistance is associated with the projection distance and based on theinter-coordinate distance; deriving three-dimensional coordinates of atleast three of the four corners of the calibration image in athree-dimensional space for the projection section to process an image,based on the projection distance and the half angle-of-view in theprojection section; deriving an angle formed by the optical axis of theprojection section and the projection target, based on thethree-dimensional coordinates; and generating the image distortioncorrection information, based on the derived angle.
 6. Acomputer-readable program which causes a computer to function as: acalibration image information generating means for generating imageinformation used to display a rectangular calibration image; aprojection means for projecting the calibration image onto a projectiontarget, based on the image information; a sensing means for sensing aregion including the projected calibration image through a sensing planeand generating sensing information; a projection area informationgenerating means for generating temporary projection area informationwhich indicates coordinates of four corners of the calibration image onthe sensing plane, based on the sensing information in such a state thatan entire portion of the calibration image is included within a sensingrange of the sensing means and under such a state that a distancebetween the projection means and the projection target is different fromthe distance in an actual operating condition, and for generating actualprojection area information which indicates the coordinates of the fourcorners of the calibration image in the sensing plane, based on thesensing information in the actual operating condition; a correctioninformation generating means for generating image distortion correctioninformation; and a distortion correction means for correcting adistortion of the image, based on the image distortion correctioninformation, wherein the correction information generating means hasfollowing functions of: deriving an inter-coordinate distance betweeneach of the coordinates of the four corners included in the temporaryprojection area information and corresponding one of the coordinates ofthe four corners included in the actual projection area information;deriving a projection distance on an optical axis of the projectionmeans between the projection means and each of the four corners of thecalibration image on the projection target, based on projection distancedata in which the inter-coordinate distance is associated with theprojection distance and based on the inter-coordinate distance; derivingthree-dimensional coordinates of at least three of the four corners ofthe calibration image in a three-dimensional space for the projectionmeans to process an image, based on the projection distance and the halfangle-of-view in the projection means; deriving an angle formed by theoptical axis of the projection means and the projection target, based onthe three-dimensional coordinates; and generating the image distortioncorrection information, based on the derived angle.
 7. An informationstorage medium storing a computer-readable program which causes acomputer to function as: a calibration image information generatingmeans for generating image information used to display a rectangularcalibration image; a projection means for projecting the calibrationimage onto a projection target, based on the image information; asensing means for sensing a region including the projected calibrationimage through a sensing plane and generating sensing information; aprojection area information generating means for generating temporaryprojection area information which indicates coordinates of four cornersof the calibration image on the sensing plane, based on the sensinginformation in such a state that an entire portion of the calibrationimage is included within a sensing range of the sensing means and undersuch a state that a distance between the projection means and theprojection target is different from the distance in an actual operatingcondition, and for generating actual projection area information whichindicates the coordinates of the four corners of the calibration imagein the sensing plane, based on the sensing information in the actualoperating condition; a correction information generating means forgenerating image distortion correction information; and a distortioncorrection means for correcting a distortion of the image, based on theimage distortion correction information, wherein the correctioninformation generating means has following functions of: deriving aninter-coordinate distance between each of the coordinates of the fourcorners included in the temporary projection area information andcorresponding one of the coordinates of the four corners included in theactual projection area information; deriving a projection distance on anoptical axis of the projection means between the projection means andeach of the four corners of the calibration image on the projectiontarget, based on projection distance data in which the inter-coordinatedistance is associated with the projection distance and based on theinter-coordinate distance; deriving three-dimensional coordinates of atleast three of the four corners of the calibration image in athree-dimensional space for the projection means to process an image,based on the projection distance and the half angle-of-view in theprojection means; deriving an angle formed by the optical axis of theprojection means and the projection target, based on thethree-dimensional coordinates; and generating the image distortioncorrection information, based on the derived angle.
 8. An imageprocessing method comprising: causing an image projection device toproject a rectangular calibration image onto a projection target,sensing a region including the projected calibration image through asensing plane, and generating first projection area information whichindicates coordinates of four corners of the calibration image in thesensing plane, when a positional relationship between the imageprojection device and the projection target becomes a first state inwhich an entire portion of the calibration image is included within asensing range; causing the image projection device to project thecalibration image onto the projection target, sensing the regionincluding the projected calibration image through the sensing plane, andgenerating second projection area information which indicates thecoordinates of the four corners of the calibration image in the sensingplane, when the positional relationship between the image projectiondevice and the projection target becomes a second state in which anentire portion of the rectangular calibration image is included withinthe sensing range, the second state being different from the firststate; deriving an inter-coordinate distance between each of thecoordinates of the four corners included in the first projection areainformation and the corresponding one of the coordinates of the fourcorners included in the second projection area information; deriving aprojection distance on an optical axis of a projected light between theimage projection device and each of the coordinates of the four cornersof the calibration image on the projection target, based on theprojection distance data in which the inter-coordinate distance isassociated with the projection distance and based on theinter-coordinate distance; deriving three-dimensional coordinates of atleast three of the four corners of the calibration image in athree-dimensional space for the image projecting device to process animage, based on the projection distance and the half angle-of-view inthe projected light; deriving an angle formed by the optical axis of theprojected light and the projection target, based on thethree-dimensional coordinates; generating image distortion correctioninformation, based on the derived angle; and correcting imagedistortion, based on the generated image distortion correctioninformation.
 9. The image processing method as defined in claim 8, whenthe angle formed by the optical axis of the projected light and theprojection target is derived, the method comprising: deriving a normalvector in the projection target, based on the three-dimensionalcoordinates; and deriving an angle formed by the optical axis of theprojected light and the projection target, based on the normal vectorand a directional vector of the optical axis of the projected light.